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- The MAILING DATE of this communication appears on the cover sheet with the correspondence address - 
Period for Reply 



A SHORTENED STATUTORY PERIOD FOR REPLY IS SET TO EXPIRE 3 MONTH(S) OR THIRTY (30) DAYS, 
WHICHEVER IS LONGER, FROM THE MAILING DATE OF THIS COMMUNICATION. 

- Extensions of time may be available under the provisions of 37 CFR 1.136(a). In no event, however, may a reply be timely filed 
after SIX (6) MONTHS from the mailing date of this communication. 

- If NO period for reply is specified above, the maximum statutory period will apply and will expire SIX (6) MONTHS from the mailing date of this communication. 

- Failure to reply within the set or extended period for reply will, by statute, cause the application to become ABANDONED (35 U.S. C. § 1 33). 
Any reply received by the Office later than three months after the mailing date of this communication, even if timely filed, may reduce any 
earned patent term adjustment. See 37 CFR 1.704(b). 

Status 

1)M Responsive to communication(s) filed on 10/28/2007 . 
2a)D This action is FINAL. 2b)ISI This action is non-final. 

3) D Since this application is in condition for allowance except for formal matters, prosecution as to the merits is 

closed in accordance with the practice under Ex parte Quayle, 1935 CD. 11, 453 O.G. 213. 

Disposition of Claims 

4) E<] Claim(s) 1 -2. 5-7, 10-12, 15-16 is/are pending in the application. 

4a) Of the above claim(s) is/are withdrawn from consideration. 

5) D Claim(s) is/are allowed. 

6) E3 Claim(s) 1 -2, 5-7, 10-12, 15-16 is/are rejected. 

7) D Claim(s) is/are objected to. 

8) D Claim(s) are subject to restriction and/or election requirement. 

Application Papers 

9) D The specification is objected to by the Examiner. 

10) Q The drawing(s) filed on is/are: a)Q accepted or b)D objected to by the Examiner. 

Applicant may not request that any objection to the drawing(s) be held in abeyance. See 37 CFR 1.85(a). 
Replacement drawing sheet(s) including the correction is required if the drawing(s) is objected to. See 37 CFR 1.121(d). 

1 1) D The oath or declaration is objected to by the Examiner. Note the attached Office Action or form PTO-152. 

Priority under 35 U.S.C. § 119 

12) D Acknowledgment is made of a claim for foreign priority under 35 U.S.C. § 119(a)-(d) or (f). 
a)D All b)D Some * c)Q None of: 

!□ Certified copies of the priority documents have been received. 

2. Q Certified copies of the priority documents have been received in Application No. . 

3. D Copies of the certified copies of the priority documents have been received in this National Stage 

application from the International Bureau (PCT Rule 17.2(a)). 
* See the attached detailed Office action for a list of the certified copies not received. 



Attachments) 

1) O Notice of References Cited (PTO-892) 4) □ Interview Summary (PTO-413) 

2) □ Notice of Draftsperson's Patent Drawing Review (PTO-948) PaP©' No(s)/Mail Date. . 

3) □ Information Disclosure Statement(s) (PTO/SB/08) 5 ) □ Notice of Informal Patent Application 

Paper No(s)/Mail Date . 6) □ Other: . 

U.S. Patent and Trademark Office 

PTOL-326 (Rev. 08-06) Office Action Summary Part of Paper No /Mail Date 20071 230 
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DETAILED ACTION 

1 . This action is responsive to the following communications: Arguments filed 04/24/2007. 

2. Claims 1 -2, 5-7, 10-12, 15-16 are pending in the case. Claims 1, 6, and 11 are the independent 
claims. Claims 3-4, 8-9, 13-14 are the cancelled claims. 

Continued Examination Under 37 CFR 1.114 

A request for continued examination under 37 CFR 11 14, including the fee set forth in 37 CFR 
1.17(e), was filed in this application after final rejection. Since this application is eligible for 
continued examination under 37 CFR 1.114, and the fee set forth in 37 CFR 1.17(e) has been 
timely paid, the finality of the previous Office action has been withdrawn pursuant to 37 CFR 
1.1 14. Applicant's submission filed on 10/28/2007 has been entered. 



Specification 

3. In light of the applicants amendment to the specification the previous objection is now considered 
moot, as there is a clear antecedent basis for the terms relied upon in the claim. 



Claim Rejections - 35 USC §103 

4. The following is a quotation of 35 U.S.C. 103(a) which forms the basis for all obviousness 
rejections set forth in this Office action: 

(a) A patent may not be obtained though the invention is not identically disclosed or described as set 
forth in section 102 of this title, if the differences between the subject matter sought to be patented and 
the prior art are such that the subject matter as a whole would have been obvious at the time the 
invention was made to a person having ordinary skill in the art to which said subject matter pertains. 
Patentability shall not be negatived by the manner in which the invention was made. 



The factual inquiries set forth in Graham v. John Deere Co., 383 U.S. 1, 148 USPQ 459 (1966), 
that are applied for establishing a background for determining obviousness under 35 
U.S.C. 103(a) are summarized as follows: 

1 . Determining the scope and contents of the prior art. 

2. Ascertaining the differences between the prior art and the claims at issue. 

3. Resolving the level of ordinary skill in the pertinent art. 
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4. Considering objective evidence present in the application indicating obviousness or 
nonobviousness. 

This application currently names joint inventors. In considering patentability of the claims under 
35 U.S.C. 103(a), the examiner presumes that the subject matter of the various claims was 
commonly owned at the time any inventions covered therein were made absent any evidence to 
the contrary. Applicant is advised of the obligation under 37 CFR 1.56 to point out the inventor 
and invention dates of each claim that was not commonly owned at the time a later invention was 
made in order for the examiner to consider the applicability of 35 U.S.C. 1 03(c) and potential 35 
U.S.C. 102(e), (f) or (g) prior art under 35 U.S.C. 103(a). 



5. Claims 1 -2, 5-7, 10-12, 15-16 are rejected under 35 U.S.C. 103(a) as being unpatentable 

over Challapali et al. (hereinafter Challapali) U.S. Publication No. 20020194006 filed Mar. 

29, 2001, in further view of Brown et al. (hereinafter Brown) U.S. Publication No. 

20040001090 filed June 27, 2002. 

The applied reference has a common assignee with the instant 
application. Based upon the earlier effective U.S. filing date of the 
reference, it constitutes prior art only under 35 U.S.C. 102(e). This 
rejection under 35 U.S.C. 103(a) might be overcome by: (1) a showing 
under 37 CFR 1.132 that any invention disclosed but not claimed in the 
reference was derived from the inventor of this application and is thus 
not an invention "by another"; (2) a showing of a date of invention for the claimed subject 
matter of the application which corresponds to subject matter disclosed 
but not claimed in the reference, prior to the effective U.S. filing 
date of the reference under 37 CFR 1.131; or (3) an oath or declaration 
under 37 CFR 1.130 stating that the application and reference are 
currently owned by the same party and that the inventor named in the 
application is the prior inventor under 35 U.S.C. 104, together with a 
terminal disclaimer in accordance with 37 CFR 1.321(c). This rejection 
might also be overcome by showing that the reference is disqualified 
under 35 U.S.C. 103(c) as prior art in a rejection under 35 U.S.C. 
103(a). See MPEP § 706.02 (1) (1) and § 706 . 02 (1) (2) . 



In regard to Independent claim 1, Challapali teaches a computer-implemented method of 
communicating emot i onal asp e cts associat e d w i th a communicat i on s e ss i on from a fir s t p e r s on to 
a se cond p e rson, sa i d m e thod comprising the steps of: 

• Receiving an emotional state indicator output from automatic on e or mor e e motional 
charact e ristic i ndicators from at least on e emotional content analyzer p e rform e d on sai d 
of a communication session from [[said]] a first person to a plurality of recipients, wherein 
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each recipient is to receive a copy of said communication session (See Para 009, 001 1 , 
0020-0025). Challapali teaches a chat system with a plurality of recipients, where each 
person has the same software and hardware and content is analyzed and presented to 
the chat participants in the form of an emoticon where the emotion is represented as a 
facial expression. 

• Eor each recipient determining an appropriate emoticon or other symbol to reflect said 
emotional state (See figure 2 and 3). Challapali teaches that each person facial 
movements are expressed through the chat system as they respond to the content in the 
chat window. Challapali lists several emotional responses (See also appendix on page 3 
and 4). 

• E or each copy to each recipient, merging said emoticon or symbol into said 
communication session copy (See Figure 2 and 3, and Para 0026-0029). Challapali 
teaches merging the emotion with the text into an emoticon that is shown to all other 
participants as the user responds (See also appendix on page 3 and 4). 

• Presenting said communication session, emoticons, and symbols to each of said plurality 
of recipients wherein each recipient receives symbols (See Para 0023- 0024). Challapali 
teaches simulating facial movements and emotions to each of the other users in the 
interface (See also figures 2 and 3). 

g e n e r a t i ng on e or mor e el e ctron i c s ymbo l s r e pr e s e nting sa i d ov e ra l l e motion, s ai d 
el ectron i c s ymbo l s b ei ng i nt e rpr e tab le by s ai d se cond p e rson; and 
pr e s e nt i ng said ele ctronic symbo l s to s ai d se cond p e r s on i n associat i on w i th s a i d 
commun i cation se ss i on. 

Challapali does not expressly teach: 

• i dent i fy i ng retrieving a profile for said first person a nd sa i d se cond p e rson; retrieving a 
plurality of profiles corresponding to each of said recipients; d e t e rmin i ng an ov e ra ll 
e mot i onal stat e for s a i d fir s t p e rson r e lativ e to sa i d first p e rson's id e nt i ty and s aid 
r e c ei v e d e mot i ona l cont e nt ana l y si s r e su l ts: determining the emoticon or symbol in a 
manner consistent with each recipient profile and presenting emoticons and symbols to a 
plurality of recipients consistent with their own profile. 

However, Brown teaches a process of relating a user profile to a response to content in a chat 

system where the system can present a symbol or emoticon based on the user profile to all of the 

recipients on the message (See Brown Para 00350051, 0063, 00650073-0074, 0082 and 105- 

106). Brown teaches the content can be provided through graphics, sounds, video and avatars 

that can make gestures. Brown and Challapali are analogous art because they both teach a 

process of expressing emotion in communications between users. They both address the 

problem of conveying emotion in email or chat where the true meaning of the context of the 

message is sometimes misunderstood. 

Accordingly, it would have been obvious to one of ordinary skill in the art at the time of 

the invention, having the teachings of Brown and Challapali in front of them, to modify the system 
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of Challapali with the system of Brown to present the emoticons of Challapali to other participants 
based on a user, profile. The motivation to combine Brown with Challapali comes from the 
suggestion in Brown that graphical elements can be used to indicate reaction responses that 
include text, graphics, icons, images and sounds, as is known in the art (See Para 0030). Further, 
Brown teaches the content of the communications may include text, icons, video, graphics and 
avatars gesturing (See Para 0033 and 0068), which is a specific example of the emoticons that 
provide facial expressions proposed in Challapali. 



With respect to dependent claim 2, Challapali teaches the computer-implemented method 
wherein said step of receiving output of emotional content analysis comprises receiving the 
results of a process selected from the group of a hand gesture recognizer, a body movement 
recognizer, a voice pitch analyzer, and a facial expression recognizer (See Challapali Para 005, 
001 1 and 0020-0025). Challapali teaches presenting content that has been analyzed by the 
system through a facial expression and presents to the other chat users the emoticon and text 
related to the response (See also figure 2 and 3 and Para 29). 

With respect to dependent claim 5, Challapali teaches a computer-implemented method wherein 
said step of generating one or more electronic symbols corresponding to said overall emotional 
state comprises generating an electronic symbol selected from the list of a text-based emoticon, a 
graphical emoticon, a text highlight method, a text size change, a text underlining method, a text 
holding method, one or more signals for a Telephone Terminal for the Deaf system, and a Braille 
code (See Para 0031, Challapali shows the list of text and graphical emotions that will be shown 
to the plurality of users). 

In regard to claims 6, 7, and 10, claims 6, 7 and 10 reflect the article of manufacture comprising 
computer readable instructions for performing the method steps of claims 1, 2 and 5, respectively, 
and are rejected along the same rationale. 
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In regard to claims 11,12, and 15, claims 11, 12 and 15 reflect the system comprising computer 
readable instructions for performing the method steps of claims 1, 2 and 5, respectively, and are 
rejected along the same rationale. 

With respect to dependent claim 16, Challapali teaches the system wherein said symbol merger 
is configured to merge a communication session selected from the group of an electronic mail 
message, an online text chat, a video conference, a online classroom, a captioned television 
broadcast, a multimedia presentation, and an open captioned meeting (See Para 005 and 017, 
chat system). 

Response to Arguments 

Applicant's arguments with respect to claims 1 -2, 5-7, 10-12, 15-16 have been considered but 
are moot in view of the new ground(s) of rejection. 

Conclusion 

The prior art made of record and not relied upon is considered pertinent to applicant's 
disclosure. 

US. Patent Publication ^^^^^^^S^^S^^^Mt hod and apparatus for 
providing multilingual translation over a network, that discloses the use of a 
profile to determine the response, sent to other users. 

Any inquiry concerning this communication or earlier communications from the examiner should 
be directed to Steven B. Theriault whose telephone number is (571) 272-5867. The examiner 
can normally be reached on M, W, F 10:00AM - 8:00 PM. 
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If attempts to reach the examiner by telephone are unsuccessful, the examiner's supervisor, 



Weilun Lo can be reached on (571) 272-4847. The fax phone number for the organization where 



this application or proceeding is assigned is 571-273-8300. 

Information regarding the status of an application may be obtained from the Patent Application 
Information Retrieval (PAIR) system. Status information for published applications may be obtained from 
either Private PAIR or Public PAIR. Status information for unpublished applications is available through 
Private PAIR only. For more information about the PAIR system, see http://pair-direct.uspto.gov. Should 
you have questions on access to the Private PAIR system, contact the Electronic Business Center (EBC) 
at 866-217-9197 (toll-free). If you would like assistance from a USPTO Customer Service Representative 
or access to the automated information system, call 800-786-9199 (IN USA OR CANADA) or 571-272- 
1000. „ y ^ 




^Steven B Theriault/ 
Patent Examiner 
Art Unit 2179 



